1. Field of the Invention
The invention relates to computer graphics, and more particularly to development of graphics hardware and software.
2. Related Art
Early Graphics Systems
Early visual systems, such as 1954's DX-50 helicopter trainer from Giravion-Dorand, used optical and mechanical systems and did not make a distinction between the database, the visual system, and the display system. The hardware-centric structure of these systems follows from the ambitious nature of their real-time performance and image quality goals relative to available hardware technology of the day. The birth of image generator technology began with Ivan Sutherland's SKETCHPAD system. The CT3, introduced ten years later by Evans & Sutherland, shaped image generator architectures still in use today. This system, first delivered to NASA, was composed of a DIGITAL EQUIPMENT CORPORATION PDP-11 front-end processor running the software, connected to non-programmable graphics hardware. A lasting innovation of this system was separating the database from the visual system, introducing a modeling tool that ran on a PDP-11/40 equipped with a calligraphic display. The CT3 was capable of 900 polygons at 25 Hz, with edge antialiasing and Gouraud shading, and defined the image generator as a database-driven system, connected to a simulation host via a hardware or software interface, with output to one or more associated display subsystems. Such a system is shown in FIG. 1. An image generator 102 is shown creating an image on the basis of data from database 104. The image is then displayed on one or more displays 106.
Scene Graph Toolkits
Since the early 1990's, general-purpose workstations have increasingly competed with dedicated hardware image generators, using sophisticated software, such as the SILICON GRAPHICS, INC. (SGI) IRIS PERFORMER, to implement real-time image generation features. For example, a scene graph was used as a graphics abstraction. A scene graph is a high-level scene and rendering description language. A scene graph is used as part of a toolkit application programming interface (API). The notion of a scene graph underlies PERFORMER and similar products, where it represents the visual database, serves as the arena of action, and is the attachment point for newly developed extensions. PERFORMER uses the scene graph abstraction to provide flexibility to application developers while retaining control of hardware- and performance-related issues. It also introduced a sample application which proved popular as a starting point for developers. PERFORMER was not the first use of scene graphs in a general-purpose graphics toolkit. PERFORMER, however, focused on performance concerns rather than developer convenience or hardware independence.
Outside the circle of IRIS PERFORMER users a common belief is that intermediate software layers degrade the performance of an application. For instance, the game industry's historical perception has been that toolkits are only for prototyping, and that applications must be rewritten before deployment. In contrast to this view, IRIS PERFORMER and PARADIGM ENTERTAINMENT's VISKIT show that an intelligent, optimizing software layer can consistently improve application performance.
The Fourth Element
The success of IRIS PERFORMER in the visual simulation industry popularized the notion of a toolkit-level API more abstract than low-level interfaces like OPENGL, yet less rigid than the interface control definition of classic image generators. This led to the introduction of a fourth element into the structure of an image generator—the visual simulation run-time—the high-level software which provided the additional features, beyond a scene graph, required to recreate the turnkey capability of classic image generators. This is shown in FIG. 2. Here, information in a database 202 is used by run-time software 204. The resulting data is used by a graphics workstation 206 to create an image for display on one or more display devices 208.
Examples of such a system include AECHELON's C-NOVA, RAYTHEON's RIGHTVIEW, THOMSON's SPACE MAGIC, the EQUIPE SIMULATION REAL TIME, MULTIGEN-PARADIGM's VEGA, and, WORMALD's WIGS, each layered above IRIS. Pairing the scene graph API and image generation run-time has improved visual simulation: decreasing risk, schedule, and cost. Developers can add features without relying on sole-source vendor enhancements, helping integrators use and retain proprietary techniques in system configurations, and better matching the power of commercial graphics hardware to needs of the image generation community.
However, advances of software technology can lead to the discovery of new barriers. This is true with the scene graph as well, where developers consistently encounter the same technical problems:
Weak Extension Model. The scene graph is a good structure for expressing hierarchical articulations, BSP definitions, level of detail (LOD) switch ranges, and similar traversal characteristics. When extended to allow application developers to attach callback functions or redefine scene graph elements, however, a problem emerges. This “tree decoration” metaphor converts a scene graph into an expression tree with critical but unstated order dependencies, surreptitiously changing the meaning of scene graph traversal with the result that any two or more extensions are likely to be incompatible.
Limited Code Reuse. It is difficult if not impossible for separate developers to build independent features that can be merged into the common scene graph due to extension model semantic conflicts. This leads integrators to develop their own visual simulation run-time environments since they lack a framework for integrating sub-components from independent software providers, resulting in multiple versions of the same base features. This troubles technical evaluators, who may know that a machine is capable of a feature, but cannot know if a particular bidder will implement it as desired—and there is no way to replace a weak implementation in one program with a strong implementation from another.
Difficulties in Integration. Even though the last 20% of the integration task seemingly requires 80% of the time and expense, there are few tools to help. Scene graphs are partially responsible, as extension model problems surface at integration time (when different sub-components first operate together) and developers cannot profile, understand, or modify the inner-workings of the scene graph. If testing reveals that a simulator “almost always” runs at frame rate, the developer is typically at a loss to know what the scene graph is doing differently to cause frame extension.
Duplication of Application State. The simulation application computes or manages the positions of objects, light sources, and other state elements. This same information must also appear in the scene graph in an opaque form due to its autonomous nature, which requires duplication of application state between the two with problems in synchronization and updating.
The scene graph is a victim of its own success: it works so well as a graphics abstraction that it has been pressed into further service as an application abstraction, which it is not. These problems are signs of an inherent structural mismatch rather than a flaw of scene graph implementations. An entirely different abstraction and corresponding data structure are needed, able to represent the innermost essence of applications. A new type of real-time graphics software is necessary, one that complements the scene graph as a higher-level counterpart for application-level concepts and concerns.
A new factor adds unprecedented urgency to these issues: lower-cost 3D graphics hardware devices have the features, quality, and performance to serve in an image generator but the corresponding software does not. In many cases, next-generation hardware offers more capability than is required for typical training applications. This does not mean, however, that building applications is becoming easier or less expensive. It will likely be harder to build an image generator from these components than from workstations, just as some integrators have found powerful general-purpose workstations more difficult to master than classic dedicated image generators before the advent of the vendor-tuned IRIS Performer API. The inventors recognized that embracing this era of astounding performance at low price-points requires addressing the portability of high-technology graphics application software, which in turn means considering applications as mobile components able to move from one hardware system to another. Software is needed that promotes distinguishing hardware capabilities by allowing hardware vendors to provide alternate implementations of standard features as a route to tuning and delivering applications.
FIGS. 3A and 3B illustrate the problem of developing game software in a manner that utilizes the features of different hardware platforms with different features. FIG. 3A illustrates a layered approach 300 to game development, where the approach requires customization to a SONY PLAYSTATION 2 platform 310. Game software 320 is tailored to take advantage of the features of platform 310. Game content 330 is adapted to meet the requirements of game software 320. Development tools 340 necessarily affect the game content 330 produced with them. FIG. 3B illustrates a similarly layered approach to game development, customized to a personal computer (PC) platform 360. As above, game software 370 is tailored to take advantage of the features of platform 360. Game content 380 is adapted to meet the requirements of game software 370. Development tools 390 necessarily affect the game content 380 produced with them. In general, a degree of customization is necessary, as suggested by the uneven boundary between hardware and game software. Moreover, this phenomenon of necessary customization tends to extend into the upper layers, as illustrated. The boundary between game software and game content is also necessarily customized; likewise, the boundary between game content and the development tools.
FIG. 4 shows an example of the development of game content. This figure shows the incorporation of various aspects of producing content 405, such as the introduction of sound 410, models 415, animation 420, and behaviors 430, through the use of a level editor 440.
An fundamentally different approach is therefore needed, relative to prior efforts, given that such prior efforts used the same underlying implementation for features and placed the “seam” between system-dependent implementations at the lower level hardware interface and scene graph APIs. These low-level approaches have an inherent weakness, as evidenced by portable 3D graphics benchmark suites. Low-level extension mechanisms restrain hardware vendors from optimizing portable applications, for instance by making it impossible to transparently insert multi-processing. They also tend to stifle innovation by requiring implementation of high-level designs using the specific approach sanctioned by the low-level API developer.
It is even difficult for hardware vendors to add new features using OpenGL's respected extension mechanism, and after having done so, application developers must be enticed to rewrite and re-release their products to incorporate these system-dependent extensions. A new software architecture is needed to address this problem by providing a higher level of abstraction that offers hardware vendors a mechanism to change the way existing, compiled application software implements features, to access the differentiations of their hardware.